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1 This study explored seven poBBible techniques for 
improving the rate of returns for' a^afuest ionnaire study with special 
concern for collecting information, (through high school ' follow-up- 
studies. It evaluated these techniques usi-ng two criteria: their 
effectiveness in improving returns and their .cost -effectiveness. The 
^resulrts suggest that a school district planning to coiiduct such a ^ 
study should orient those who will be involved prior to their lea wing* 
school. It further suggests that prior \o receiving the instrument 
those involved in the stud/ should be contacted by telephone, : to 
elicit their cooperation, particularly when the geographic area is 
smal?l arid local calls can be made.. The writer co^fcludes thai there is 

need to study more complex methods ^or improving the rate of 
returns of follow-up studies, with a strong probability that 
telephoning ahead, and using a follow-up that includes a second copy 
of the ins'trument and a second cover letter, w^O;uld produce both 
grea,ter cost effectiveness^ and a higher rate of return. 
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School guidance counselors share with athers in t^he educational. 
community a reliance ^pon questionnaires fpr the collect loin of infor©atrf.oh/. 



return: response rates ^s.IoU 



* They also share the problem o£ a poor rate pf return: r< 
ias 10 to 30 percent are not unnmnon. In an attempt to improve these 



return levels, many Educator^ mVe^ been forced to create complex «search 

2 ' . • * . i/ 

proceduies. Today t)iere exists a b<^y of research suggesting ilays to ; ^ 

. 3 

Iniprove the rate of returns of questionnaires. Yet, even vit4i this body 
of writing, the process of using questionnaires seems to be as much one of 

art as of science, with little definitive data existing to guide those 

* *• ' * .' i' * , ' ' " 

who seek productive procedures. 

This article reports the results of a study of seven techniques 
- expected to maximise th^ retus^s of a survey of high^school^graduates and 
^^drop-ou±8.; It Evaluates these techniques using two criteria: their 

effece^vei^ess in improving returns and their cost-effectiveness. Guidance 
couns^rs Wfto would qonduct follow-up studies involving high ^hool and ^ 
ftost-hig^^hool age subjects may find the results helpful as they design 
their da|^K#thering process. 
>r ^ Purpose of the Study 

JjQ , The study\rew from the desire of a large mid-western school district 

to conduct a follow-up of those who had graduated or dropped 6ut of their 
high schools. Since the follow-up waa to be used to develop » computerised 



dat« bank, and WiDuld rely on ^anpllng techniques, thete vas a perceiWd need 
to* ensure a very high (90 to 95 percent), Va^ie pf return . Based on thi? 
)ieed, ia pilot j^as conducted of a nuvber o£ pokai-ble approaches .to the coll^ 
ection of' data using questionnaires. AJ.so, sitnce school district* 6ann«t 
lightly ignore cost^ an effort was inade to identify the cost-effectiveness. 



Of each ^echnlque. 



- ^ Design of the -Study i 

' •••• » ' .• . • ■ . • ^ t \- • ' . 

* -In using a questionnaire in a follov-up study, the usual procedure is 
to s^nd itn instrument and a cover letter to each participant explainl^ng 
' the purposes of the study* A number of additional step,s cbuld be added, 
with the logical expectation that the rate of return might Improve. For 



example > a cdphseior could "telephone ahead" to et^jSure* that the person 
understood the ptirpose of th4 follow-up and^ would agr^e to particip^ate . 
Various "forms of follow-ups could be used; including the use of the tele- 
phone » postcards, and seco^/^ppies of iiistruroentSA and letters, if the 
person failed to return arii^^^^nment . There could^ even be an offer to , 
pay those who reSj)ondf5d. ^ ^ ■'^"'>^' ' , 

Based on these .possiibllities, and recognizing that there are others, 
seven different techniques for gathering questionnaire informatiita were 
identified. In the spring of 1973, sev^jpty participants from each of the 
four high schools in the district wer^ chosen at random from a list of Seniors 
of the previous year. Ten from each school were assigned to a. group, making 
a total of forty in' each of the iseven groups* ^^ch group was assigned one 
of the collection techniques described below: ^ 



Group One: provided the BASg |%^|p9^ which all other groups and 



techniques were compared. This g>Oug received a letter and instrument with 
no follow-up or contact prior repeii^ing the instrument and letter. 



• / 
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Group Two: nenbers of this gr«Up were offered $3.00 for cpmpletion 

of the instruntnt; no follow-up was provided for t;hose who did not respond. , 

Group ^»ree: neinbers of this group were offered $2..00 for comple - 

> 

tlon of the instrunent; again no follow-up^ vas provided. 

Group ^our: this group was contacted by telephone and asked to^ 
participate in thjs study prior, to receiving an instrument. No follow-up 
was provided for %hose who did nbt respond. 

■ ■ ■ : 'J. 

Group Five: received a telephone call follow-up if they did not 
return an instrument one week after receiving it. 

firoup Six;, received a post card follow-up if they did not return an 
instrument one .week after receiving it. 

Group Seven: received a second copy of ' the instrumeat and a second ^ ~. 
cover letter if they did not return an instrument one week after receiving 
it. 

Responses were: accepted from the 280 subjects for a period of three weeks 

to simulate the realities of the graduate follow-up process that would be ^ 

4 ■ 

used later by the district. 

<. 

The instrument used in the pilot included questions to elicit data 
concerning activities in school, since leaving ichool, and plans for the 
future. ^ ■ , 

Limitations of the Study . 
The study c«»Dpared only a few of the possible methods for Improving 
the rate of return for questionnaire studies. F6r example, no effort was 
• made to explore all of the follow-up 'procedures "possible when the person ^ 
failed to teturn an instrument. It also did not explore complex combination* 
such aj contacting the participants via telfephone prior to send ij|' out the 
. Instruaent, and then sending a follow-up letter or letters JJ^^jt^o di<i 



not respond, 

A second limitation cpncerns the use ofl cost estimates in the stu^. 
Since the school district conducting the study was able| to us6 sophisticated 
systems for typing and in-house printing services, the cost figures gen-~ 
erated would be difficult ,to transfer to other settings that must rely 
on qore traditional and more expensive duplication systems. ^ 



y . . / Results of the Study/ 

The resuItB of the study to improve fate of returns are sunnarlsed 
in Tible I. / 

^ 



/ lAsert Table I here 



Fiylng for the return of the instEument was , as expecte^^j the ifpi^^ 

successful' of the techniques piloted. .When $3.00 wa& of fered/ for a^ ^cobh^( 

■ / ' ' ' ''■ , ■ - ■ ' ' t ' 

pleted question^ire/ the response rate, after three weeks, was 73 percent ^ 

^(76 percent after four we^ks, or approximately 30 to 40 percent high^i^ 

- * * ' »■ " . / . ■ ' ■ 

than a procedure that, sent questloni^ires and used no follow-up for those 
who djLd not respond), ^^^^ver, even financial reward by itself seemed to 
be an inadequate fa ct^^n motivating «!! people to return a questionnaire. 

None of the teclmiques generated a return rate at the. level originally 
sought by the school district. It was concluded that it would be very 
difficult to provi/le an extremely high rate of return for a high school 
follow-up study without contacting students while they .ar^ still in school 
to explain the* project and elicit their cooperation. This approach would 
ensure that stuaents^ understood the study and may encourage them to respond' 
when contacted /after graduation. Since this was the original intent of the 
dlstr^ict, the blan seema Justified. i ^ 
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Results suggest tl^t the use of the telephone with follow-up studies 
should be encouraged. The BASE technique return (Groups One) was. lioproved 
significantly (by' 30 percent) by telephoning and explaining the project. . 

■ , , - ' ' ■ ■ / 

Telephoning those who did not return an, Instrument (Group. Five) provided/ 

' . , ■ ' ' ' '' 

ah"* Inprovenent 6f approxlioately 20 percent over the BASE group i 

... 9 

Some type of follow^^up of those yho did not respoifd Is vital. Each' 

of the follow-up techniques piloted Improved the rij^te of return, with the 

use of a post card the least' effect Ive, an4 the use of a second letter a>nd 

/ » -. ^ . 

Instrument or a telephone call each Improving the return rate by approx-' 

imately 20 percent. / ' h 

. ' ' ' / 

Cbst/ Considerations 

Each technique requires certain BASE COSTS. These Include the cost 

for typing y duplicating the Instrument and cover letters » an4 postage. 

These BASE COSTS we^e Identified as $152.25 or apptoxlmately^21.75. for 

each of the seve'n/groups/technlques used In the study. 

In addltlon/y each alternative ^method for collecting data requires ^ 

certain SPECIAL COSTS » as follows: ' ^ 

Group Two:' $3.00 paid to each respondent. 

Group ^r«e: $2.00 paid to each respondent. 

Grouf/Four: telephoning AO^plus people prior to sending out the 

-/ 

Instrumeii 

/ ' ' 

Group Five: telephoning those who did not respond after one week. 

SrOup Six: Printing and mailing a post card to those who did not 

^ resp^ond after one week. 

Group Seven: printing and mailing a second letter and Instrument to 

l^hose who did not; repond after one week. 

Together, BASE, and , SPECIAL COSTS provide a basis for Identifying the 



TOTAL COSTS of ' each tecfmlquCi These COSTS are s^own I'n Table 2. 

. / 
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Yet cost in itself is probably not the final consideration in the 

selection, of the ^foptfr technique for collecting inforaation. Hhat we 

• ,. / . ■ " ' ^ 

seek is not the cheapest change w6 can nirke ii| the BASE technique, but 
» . . ' • * • ^ ^ \ . ' • .. . ^' 

the one that produces the greatest loprovement in^return per cost in^ 

V 

crease. To put it another way, we nust identify the cos t/ef feet iyene'ss 

. . ,y . , . ^ ^ 

of the various techniques and select the one that ne^ts the specific \ 

• . ^ , ^ . ' ■' 'I 

needs of the counselor conducting the study. ^ - ^ 

To coiDpute a cost/effectivenes^ ratio for ea9h of the techniques 
the TOTAL COST of the technique was divided by the^nu»ber of returns 
it generated. The result is expressed as « J'cost-per-return" ratio and 
is plotted in Figure 1. 

— ^ ' / . ^ . ■ 

Insert Figure 1 here 



We can now begin to identify two possible apprekches to the selection 
of a technique. to gather infomationt First^assune that the goal is to 
condyct a^tudy that nust produce a high rate-of -return with cpst for 
the collection of data a secondary cons idez;at! ion. The results of t^is 
study suggest, that high rates of. return will produced f iron' those 
techniques clustering to th^ "right" on Figure 1 (techniques 2, 3, and 4). 

Assune that in a second school district, the goal is to keep cost 
for data col^lectioii down while seeking the highest possible rate of 
returns. The least expensive ijatfonsatipn collecting techniques be 

•elected- from those clustering, near the "bottoo^^f Figure ^ 5). 



• 



* 

Techniques offering the J)e8t cost/ef fectivejiess ratio (or, iihose producing 
the no8t returns for each dollar spent) will cluster near the 'Uower 
right hand corner" of Pjitgure 1 (techniques 5, 7, and 4). I 

In sunnary, this study explored a few df the possible techniques .for 
l^rbving the rate of returns for a questionnaire study with special 
.cmicern for collecting information through high school follow-up studies. 
The results suggest* that a school district planning to conduct such a 
study should orientate those who will be involved prior to their leaving 
school. It further suggests that prior to receiving the instrument those 
involved in the study should be cpntacted by .telephone^ to elicit their 
cooperation, particularly when the geographic area is small and local 
calls* can be made. The writer concludes that there is a need to study more 
complex methods for improving the rate of returns of follow-up studies, 
with a strong probability that telephoning ahead, and using a follow-up 
that includes a second' copy of the instrument and a second cover lette^ 
would produce both a cost/effective and a high rate of -return. 
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i-^ Table 1 

The Return Rate of Alternative Techniques 

Technique De«crlptlon X at Return* After . 

Three Weeks 

1 ; BASE Group; ifo treatment ^ 38^1^ 

6 lost card 'follow-up 50X 

7 2nd letter an^ Inatrinient fallov-up '58X 

' ,5 Telephone fol( low-up 5.8X ' ^ 

3 Paying $2.00 for returning instrument 65X^ 

4 • Telephoning ahtead / 68X 

^ 2 Paying $3.00 for returnljag Instrument 73X 

' ** ' ^ 

N aent t^o each/^roup "40 

' ' - ■ ' ' J. , * t ■ 

' ■■ " *• <•■•■■ - . • , 
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Tablft 2 ♦ - 
^^^Costa Associated with Each Technique 



Costs 



Description / BASE SPECIAL TOTAL Per 'Return 

- BASE 6rout>; no treatment $21.75 $ O.OQ* $21.75 $1.55 

Follow-up by telephone " 6.00 27.75' 1.40 

Folloy-up by post card ~^ 8.85 30.60 1.7,3 

PoIIof»-up by 2nd letter/inst. " 12.85 34.68 1.70 

Telephoning ahead " > 30.00 51.75 > 1.89 

t^j $2, for conpletlng Instrument " ' 42.00 63.75 a. 04 

Pay $3 for coo^ei^g .instruBent " 84.00. 105.75^ 3.80 



* .- '■ 



*- Figure 1 

^omp^rUon of Techniques 
by Cost/Effectiveness lUtlp 
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